Encoding Sequential Information in Vector Space Models of Semantics: Comparing Holographic Reduced Representation and Random Permutation

Encoding information about the order in which words typically appear has been shown to improve the performance of high-dimensional semantic space models. This requires an encoding operation capable of binding together vectors in an order-sensitive way, and efficient enough to scale to large text corpora. Although both circular convolution and random permutations have been enlisted for this purpose in semantic models, these operations have never been systematically compared. In Experiment 1 we compare their storage capacity and probability of correct retrieval; in Experiments 2 and 3 we compare their performance on semantic tasks when integrated into existing models. We conclude that random permutations are a scalable alternative to circular convolution with several desirable properties

Communicating risks and benefits to cardiology patients

Trying to explain potential outcomes, and their likelihoods, is a challenge. Indeed, patients who just had a stent procedure remember few of the related risks and benefits [1]. Patients are often unfamiliar with the terminology, vary widely in health status, numeracy, health literacy, and information preferences. Complicating things further is aiming to ensure that patients understand the “material risks” for them as individuals: a matter not just of probability, but also the impact it could have on them personally

How well did experts and laypeople forecast the size of the COVID-19 pandemic?

Throughout the COVID-19 pandemic, social and traditional media have disseminated predictions from experts and nonexperts about its expected magnitude. How accurate were the predictions of 'experts'-individuals holding occupations or roles in subject-relevant fields, such as epidemiologists and statisticians-compared with those of the public? We conducted a survey in April 2020 of 140 UK experts and 2,086 UK laypersons; all were asked to make four quantitative predictions about the impact of COVID-19 by 31 Dec 2020. In addition to soliciting point estimates, we asked participants for lower and higher bounds of a range that they felt had a 75% chance of containing the true answer. Experts exhibited greater accuracy and calibration than laypersons, even when restricting the comparison to a subset of laypersons who scored in the top quartile on a numeracy test. Even so, experts substantially underestimated the ultimate extent of the pandemic, and the mean number of predictions for which the expert intervals contained the actual outcome was only 1.8 (out of 4), suggesting that experts should consider broadening the range of scenarios they consider plausible. Predictions of the public were even more inaccurate and poorly calibrated, suggesting that an important role remains for expert predictions as long as experts acknowledge their uncertainty

How well did experts and laypeople forecast the size of the COVID-19 pandemic?

Throughout the COVID-19 pandemic, social and traditional media have disseminated predictions from experts and nonexperts about its expected magnitude. How accurate were the predictions of 'experts'-individuals holding occupations or roles in subject-relevant fields, such as epidemiologists and statisticians-compared with those of the public? We conducted a survey in April 2020 of 140 UK experts and 2,086 UK laypersons; all were asked to make four quantitative predictions about the impact of COVID-19 by 31 Dec 2020. In addition to soliciting point estimates, we asked participants for lower and higher bounds of a range that they felt had a 75% chance of containing the true answer. Experts exhibited greater accuracy and calibration than laypersons, even when restricting the comparison to a subset of laypersons who scored in the top quartile on a numeracy test. Even so, experts substantially underestimated the ultimate extent of the pandemic, and the mean number of predictions for which the expert intervals contained the actual outcome was only 1.8 (out of 4), suggesting that experts should consider broadening the range of scenarios they consider plausible. Predictions of the public were even more inaccurate and poorly calibrated, suggesting that an important role remains for expert predictions as long as experts acknowledge their uncertainty

Encoding Sequential Information in Semantic Space Models: Comparing Holographic Reduced Representation and Random Permutation

Circular convolution and random permutation have each been proposed as neurally plausible binding operators capable of encoding sequential information in semantic memory. We perform several controlled comparisons of circular convolution and random permutation as means of encoding paired associates as well as encoding sequential information. Random permutations outperformed convolution with respect to the number of paired associates that can be reliably stored in a single memory trace. Performance was equal on semantic tasks when using a small corpus, but random permutations were ultimately capable of achieving superior performance due to their higher scalability to large corpora. Finally, “noisy” permutations in which units are mapped to other units arbitrarily (no one-to-one mapping) perform nearly as well as true permutations. These findings increase the neurological plausibility of random permutations and highlight their utility in vector space models of semantics

Do colored cells in risk matrices affect decision-making and risk perception? Insights from randomized controlled studies

Risk matrices communicate the likelihood and potential impact of risks and are often used to inform decision-making around risk mitigations. The merits and demerits of risk matrices in general have been discussed extensively, yet little attention has been paid to the potential influence of color in risk matrices on their users. We draw from fuzzy-trace theory and hypothesize that when color is present, individuals are likely to place greater value on reducing risks that cross color boundaries (i.e., the boundary-crossing effect), leading to sub-optimal decision making. In two randomized controlled studies, employing forced-choice and willingness-to-pay measures to investigate the boundary-crossing effect in two different color formats for risk matrices, we find preliminary evidence to support our hypotheses that color can influence decision making. The evidence also suggests that the boundary-crossing effect is only present in, or is stronger for, higher numeracy individuals. We therefore recommend that designers should consider avoiding color in risk matrices, particularly in situations where these are likely to be used by highly numerate individuals, if the communication goal is to inform in an unbiased way

Tracing Shifting Conceptual Vocabularies Through Time

This paper presents work in progress on an algorithm to track and identify changes in the vocabulary used to describe particular concepts over time, with emphasis on treating concepts as distinct from changes in word meaning. We apply the algorithm to word vectors generated from Google Books n-grams from 1800-1990 and evaluate the induced networks with respect to their flexibility (robustness to changes in vocabulary) and stability (they should not leap from topic to topic). We also describe work in progress using the British National Biography Linked Open Data Serials to construct a “ground truth” evaluation dataset for algorithms which aim to detect shifts in the vocabulary used to describe concepts. Finally, we discuss limitations of the proposed method, ways in which the method could be improved in the future, and other considerations.Cambridge Centre for Digital Knowledge, University of Cambridg

The Idea of Liberty, 1600-1800: A Distributional Concept Analysis.

This article uses computational and statistical methods for analyzing the concept of liberty 1600-1800. Based on a bespoke set of tools for parsing conceptual structures it contributes to the literature on the concept of liberty and engages with the thesis concerning negative liberty first put forward by Isaiah Berlin and subsequently modified by Quentin Skinner

Correction: Creating genetic reports that are understood by nonspecialists: a case study

An amendment to this paper has been published and can be accessed via a link at the top of the paper

How do the UK public interpret COVID-19 test results? Comparing the impact of official information about results and reliability used in the UK, USA and New Zealand: a randomised controlled trial.

OBJECTIVES: To assess the effects of different official information on public interpretation of a personal COVID-19 PCR test result. DESIGN: A 5×2 factorial, randomised, between-subjects experiment, comparing four wordings of information about the test result and a control arm of no additional information; for both positive and negative test results. SETTING: Online experiment using recruitment platform Respondi. PARTICIPANTS: UK participants (n=1744, after a pilot of n=1657) quota-sampled to be proportional to the UK national population on age and sex. INTERVENTIONS: Participants were given a hypothetical COVID-19 PCR test result for 'John' who was presented as having a 50% chance of having COVID-19 based on symptoms alone. Participants were randomised to receive either a positive or negative result for 'John', then randomised again to receive either no more information, or text information on the interpretation of COVID-19 test results copied in September 2020 from the public websites of the UK's National Health Service, the USA's Centers for Disease Control, New Zealand's Ministry of Health or a modified version of the UK's wording. Information identifying the source of the wording was removed. MAIN OUTCOME MEASURES: Participants were asked 'What is your best guess as to the percent chance that John actually had COVID-19 at the time of his test, given his result?'; questions about their feelings of trustworthiness in the result, their perceptions of the quality of the underlying evidence and what action they felt 'John' should take in the light of his result. RESULTS: Of those presented with a positive COVID-19 test result for 'John', the mean estimate of the probability that he had the virus was 73% (71.5%-74.5%); for those presented with a negative result, 38% (36.7%-40.0%). There was no main effect of information (wording) on these means. However, those participants given the official information from the UK website, which did not mention the possibility of false negatives or false positives, were more likely to give a categorical (100% or 0%) answer (UK: 68/343, 19.8% (15.9%-24.4%); control group: 42/356, 11.8% (8.8%-15.6%)); the reverse was true for those viewing the New Zealand (NZ) wording, which highlighted the uncertainties most explicitly (20/345: 5.8% (3.7%-8.8%)). Aggregated across test result (positive/negative), there was a main effect of wording (p<0.001) on beliefs about how 'John' should behave, with those seeing the NZ wording marginally more likely to agree that 'John' should continue to self-isolate than those viewing the control or the UK wording. The proportion of participants who felt that a symptomatic individual who tests negative definitely should not self-isolate was highest among those viewing the UK wording (31/178, 17.4% (12.5%-23.7%)), and lowest among those viewing the NZ wording (6/159, 3.8% (1.6%-8.2%)). Although the NZ wording was rated harder to understand, participants reacted to the uncertainties given in the text in the expected direction: there was a small main effect of wording on trust in the result (p=0.048), with people perceiving the test result as marginally less trustworthy after having read the NZ wording compared with the UK wording. Positive results were generally viewed as more trustworthy and as having higher quality of evidence than negative results (both p<0.001). CONCLUSIONS: The public's default assessment of the face value of both the positive and negative test results (control group) indicate an awareness that test results are not perfectly accurate. Compared with other messaging tested, participants shown the UK's 2020 wording about the interpretation of the test results appeared to interpret the results as more definitive than is warranted. Wording that acknowledges uncertainty can help people to have a more nuanced and realistic understanding of what a COVID-19 test result means, which supports decision making and behavioural response. PREREGISTRATION AND DATA REPOSITORY: Preregistration of pilot at osf.io/8n62f, preregistration of main experiment at osf.io/7rcj4, data and code available online (osf.io/pvhba)